Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add: standalone plugin for evaluating dependencies with a graph #774

Merged
merged 21 commits into from
Mar 3, 2025
Merged

Add: standalone plugin for evaluating dependencies with a graph #774

merged 21 commits into from
Mar 3, 2025

Conversation

NiklasHargarter
Copy link
Contributor

@NiklasHargarter NiklasHargarter commented Dec 11, 2024 •
edited
Loading

What

Adds a standalone plugin for evaluating script dependencies with a directed networkx graph.

checks for:

  • duplicate dependencies (a script declaring dependency on other script multiple times)
  • checking for cyclic dependencies
  • checking for missing dependencies
  • checking for cross feed dependencies (community script dependence on enterprise script). It is differentiated between dependencies that are behind a enterprise feed gate and those that are not.
  • category order
  • dependency on deprecated script

included functionality of normal plugins

  • dependencies (not included the subdirectory placement warning)
  • dependency_category_order (missing ACT_SCANNER error)
  • deprecated_dependency (i use the helper pattern regex not the one from the plugin)

Output

python logging levels for system information (error, warning, info)
normal additive verbosity up to -vv for result output.

Feed options

  • 21.04 (21.04 + common)
  • 22.04 (22.04 + common)
  • common
  • full (21.0 + 22.04 + common)

example call:
poetry run troubadix-dependency-graph ~/gb/vulnerability-tests/nasl --feed full --log info -vv

Execution Time

locally ~13 seconds

Why

When checking dependencies, it makes sense to analyse the whole feed, rather than just working on changed scripts. And working on the whole feed is easier with a standalone plugin that doesn't have to adhere to the Troubadix structure.

References

Checklist

  • Tests

@github-actions GitHub Actions
Copy link

github-actions bot commented Dec 11, 2024 •
edited
Loading

Conventional Commits Report

Type Number
Changed 16
Added 5

🚀 Conventional commits found.

@codecov Codecov
Copy link

codecov bot commented Dec 11, 2024 •
edited
Loading

Codecov Report

Attention: Patch coverage is 91.84549% with 19 lines in your changes missing coverage. Please review.

Project coverage is 80.91%. Comparing base (23d065a) to head (cc8d77e).
Report is 23 commits behind head on main.

Files with missing lines Patch % Lines
...alone_plugins/dependency_graph/dependency_graph.py 86.72% 9 Missing and 6 partials ⚠️
...adix/standalone_plugins/dependency_graph/checks.py 95.65% 1 Missing and 1 partial ⚠️
...oubadix/standalone_plugins/dependency_graph/cli.py 91.30% 1 Missing and 1 partial ⚠️
Additional details and impacted files 
@@            Coverage Diff             @@
##             main     #774      +/-   ##
==========================================
+ Coverage   79.84%   80.91%   +1.06%     
==========================================
  Files          87       91       +4     
  Lines        3027     3212     +185     
  Branches      591      608      +17     
==========================================
+ Hits         2417     2599     +182     
- Misses        462      463       +1     
- Partials      148      150       +2

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@github-actions GitHub Actions
Copy link

github-actions bot commented Dec 12, 2024 •
edited
Loading

Dependency Review

The following issues were found:
  • ✅ 0 vulnerable package(s)
  • ✅ 0 package(s) with incompatible licenses
  • ✅ 0 package(s) with invalid SPDX license definitions
  • ⚠️ 2 package(s) with unknown licenses.
See the Details below.

Snapshot Warnings

⚠️: No snapshots were found for the head SHA cc8d77e.
Ensure that dependencies are being submitted on PR branches and consider enabling retry-on-snapshot-warnings. See the documentation for more information and troubleshooting advice.

License Issues

poetry.lock

PackageVersionLicenseIssue Type
networkx3.4.2NullUnknown License

pyproject.toml

PackageVersionLicenseIssue Type
networkx^ 3.4.2NullUnknown License
Allowed Licenses: 0BSD, AGPL-3.0-or-later, Apache-2.0, BlueOak-1.0.0, BSD-2-Clause, BSD-3-Clause-Clear, BSD-3-Clause, BSL-1.0, CAL-1.0, CC-BY-3.0, CC-BY-4.0, CC-BY-SA-4.0, CC0-1.0, EPL-2.0, GPL-2.0-only, GPL-2.0-or-later, GPL-2.0, GPL-3.0-or-later, ISC, LGPL-2.0-only, LGPL-2.0-or-later, LGPL-2.1-only, LGPL-2.1-or-later, LGPL-2.1, LGPL-3.0-only, LGPL-3.0, LGPL-3.0-or-later, MIT, MIT-CMU, MPL-1.1, MPL-2.0, OFL-1.1, PSF-2.0, Python-2.0, Python-2.0.1, Unicode-DFS-2016, Unlicense, Zlib, ZPL-2.1

OpenSSF Scorecard

PackageVersionScoreDetails
pip/networkx 3.4.2 🟢 4.9
Details
CheckScoreReason
Code-Review🟢 9Found 29/30 approved changesets -- score normalized to 9
Maintained🟢 1030 commit(s) and 18 issue activity found in the last 90 days -- score normalized to 10
Dangerous-Workflow⚠️ 0dangerous workflow patterns detected
CII-Best-Practices⚠️ 0no effort to earn an OpenSSF best practices badge detected
Token-Permissions⚠️ 0detected GitHub workflow tokens with excessive permissions
Binary-Artifacts🟢 10no binaries found in the repo
License🟢 9license file detected
Vulnerabilities🟢 100 existing vulnerabilities detected
Pinned-Dependencies⚠️ 0dependency not pinned by hash detected -- score normalized to 0
Fuzzing🟢 10project is fuzzed
Branch-Protection⚠️ 0branch protection not enabled on development/release branches
Packaging🟢 10packaging workflow detected
Signed-Releases⚠️ -1no releases found
Security-Policy⚠️ 0security policy file not detected
SAST⚠️ 0SAST tool is not run on all commits -- score normalized to 0
pip/pontos 25.3.0 🟢 7.5
Details
CheckScoreReason
Dependency-Update-Tool🟢 10update tool detected
Maintained🟢 1030 commit(s) and 0 issue activity found in the last 90 days -- score normalized to 10
Code-Review🟢 5Found 2/4 approved changesets -- score normalized to 5
Dangerous-Workflow🟢 10no dangerous workflow patterns detected
Token-Permissions⚠️ 0detected GitHub workflow tokens with excessive permissions
Pinned-Dependencies🟢 3dependency not pinned by hash detected -- score normalized to 3
Binary-Artifacts🟢 10no binaries found in the repo
CII-Best-Practices⚠️ 0no effort to earn an OpenSSF best practices badge detected
Vulnerabilities🟢 100 existing vulnerabilities detected
Packaging🟢 10packaging workflow detected
License🟢 10license file detected
SAST🟢 10SAST tool is run on all commits
Signed-Releases🟢 85 out of the last 5 releases have a total of 5 signed artifacts.
Security-Policy🟢 10security policy file detected
Fuzzing⚠️ 0project is not fuzzed
Branch-Protection🟢 8branch protection is not maximal on development and all release branches
CI-Tests🟢 1012 out of 12 merged PRs checked by a CI test -- score normalized to 10
Contributors🟢 6project has 2 contributing companies or organizations -- score normalized to 6
pip/networkx ^ 3.4.2 🟢 4.9
Details
CheckScoreReason
Code-Review🟢 9Found 29/30 approved changesets -- score normalized to 9
Maintained🟢 1030 commit(s) and 18 issue activity found in the last 90 days -- score normalized to 10
Dangerous-Workflow⚠️ 0dangerous workflow patterns detected
CII-Best-Practices⚠️ 0no effort to earn an OpenSSF best practices badge detected
Token-Permissions⚠️ 0detected GitHub workflow tokens with excessive permissions
Binary-Artifacts🟢 10no binaries found in the repo
License🟢 9license file detected
Vulnerabilities🟢 100 existing vulnerabilities detected
Pinned-Dependencies⚠️ 0dependency not pinned by hash detected -- score normalized to 0
Fuzzing🟢 10project is fuzzed
Branch-Protection⚠️ 0branch protection not enabled on development/release branches
Packaging🟢 10packaging workflow detected
Signed-Releases⚠️ -1no releases found
Security-Policy⚠️ 0security policy file not detected
SAST⚠️ 0SAST tool is not run on all commits -- score normalized to 0

Scanned Files

  • poetry.lock
  • pyproject.toml

@NiklasHargarter NiklasHargarter force-pushed the dependency_graph branch 2 times, most recently from f43bc1f to 63c0f64 Compare January 20, 2025 08:45
mbrinkhoff
mbrinkhoff requested changes Jan 20, 2025
View reviewed changes
amy-gb
amy-gb reviewed Jan 23, 2025
View reviewed changes
Copy link
Contributor

@amy-gb amy-gb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tried it locally and it works great! Left a couple of small comments but I will check back when Max has finished his review

@mbrinkhoff mbrinkhoff mentioned this pull request Jan 29, 2025
Merged
1 task
@NiklasHargarter NiklasHargarter marked this pull request as ready for review February 11, 2025 11:16
@NiklasHargarter NiklasHargarter requested a review from a team as a code owner February 11, 2025 11:16
amy-gb
amy-gb previously approved these changes Feb 26, 2025
View reviewed changes
Copy link
Contributor

@amy-gb amy-gb left a comment •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Max asked me to do a final review of this and I think it looks ready! Ran it locally and it works, it seems like all the review comments have been resolved as well. The cleanup the file/directory methods is good too. I suggested some small spelling error fixes, and once the merge conflict is fixed, looks good 👍

NiklasHargarter and others added 15 commits March 3, 2025 11:22
@NiklasHargarter
Add: standalone plugin for evaluating dependencies with a graph
56d3db1
@NiklasHargarter
Change: dependency_graph codeql mixed returns fix
13fb6a7
@NiklasHargarter
Change: Revert "Change: dependency_graph codeql mixed returns fix"
c33b6e3 
This reverts commit 2d9a86f.
@NiklasHargarter
Add: standalone dependency_graph unittests
25aeecd
@NiklasHargarter
Add: include when a dependency is gated in the dependency graph
3be9e46
@NiklasHargarter
Change: dependency_graph minor restructure and changes
109ea7b
@NiklasHargarter
Add: category order, deprecated dependencies to graph standalone plugin
de70954
@NiklasHargarter
Change: dependency_graph checks result structure and output
c980a56
@NiklasHargarter
Change: various refactors in dependency graph standalone
f2bcd32
@NiklasHargarter
Change: correct dependency graph unittests not fixed in last commit
332a237
@NiklasHargarter
Change: dependency graph minor refactors
8326517
@NiklasHargarter
Change: dependency graph rename variables
288c3dc
@NiklasHargarter
Add: dir and file type for argparser that only allow existing
9c4f3e6
@NiklasHargarter
Change: reuse method for splitting dependencies
226f2b5
@mbrinkhoff @NiklasHargarter
Change: Cleanup dependency_graph
a2c65e0
@amy-gb amy-gb merged commit a1a048b into main Mar 3, 2025
13 checks passed
@amy-gb amy-gb deleted the dependency_graph branch March 3, 2025 12:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@mbrinkhoff mbrinkhoff mbrinkhoff left review comments

@amy-gb amy-gb amy-gb approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@NiklasHargarter @mbrinkhoff @amy-gb